16 research outputs found
Collaborative Artificial Intelligence Algorithms for Medical Imaging Applications
In this dissertation, we propose novel machine learning algorithms for high-risk medical imaging applications. Specifically, we tackle current challenges in radiology screening process and introduce cutting-edge methods for image-based diagnosis, detection and segmentation. We incorporate expert knowledge through eye-tracking, making the whole process human-centered. This dissertation contributes to machine learning, computer vision, and medical imaging research by: 1) introducing a mathematical formulation of radiologists level of attention, and sparsifying their gaze data for a better extraction and comparison of search patterns. 2) proposing novel, local and global, image analysis algorithms. Imaging based diagnosis and pattern analysis are high-risk Artificial Intelligence applications. A standard radiology screening procedure includes detection, diagnosis and measurement (often done with segmentation) of abnormalities. We hypothesize that having a true collaboration is essential for a better control mechanism, in such applications. In this regard, we propose to form a collaboration medium between radiologists and machine learning algorithms through eye-tracking. Further, we build a generic platform consisting of novel machine learning algorithms for each of these tasks. Our collaborative algorithm utilizes eye tracking and includes an attention model and gaze-pattern analysis, based on data clustering and graph sparsification. Then, we present a semi-supervised multi-task network for local analysis of image in radiologists\u27 ROIs, extracted in the previous step. To address missing tumors and analyze regions that are completely missed by radiologists during screening, we introduce a detection framework, S4ND: Single Shot Single Scale Lung Nodule Detection. Our proposed detection algorithm is specifically designed to handle tiny abnormalities in lungs, which are easy to miss by radiologists. Finally, we introduce a novel projective adversarial framework, PAN: Projective Adversarial Network for Medical Image Segmentation, for segmenting complex 3D structures/organs, which can be beneficial in the screening process by guiding radiologists search areas through segmentation of desired structure/organ
Semi-supervised multi-task learning for lung cancer diagnosis
Early detection of lung nodules is of great importance in lung cancer
screening. Existing research recognizes the critical role played by CAD systems
in early detection and diagnosis of lung nodules. However, many CAD systems,
which are used as cancer detection tools, produce a lot of false positives (FP)
and require a further FP reduction step. Furthermore, guidelines for early
diagnosis and treatment of lung cancer are consist of different shape and
volume measurements of abnormalities. Segmentation is at the heart of our
understanding of nodules morphology making it a major area of interest within
the field of computer aided diagnosis systems. This study set out to test the
hypothesis that joint learning of false positive (FP) nodule reduction and
nodule segmentation can improve the computer aided diagnosis (CAD) systems'
performance on both tasks. To support this hypothesis we propose a 3D deep
multi-task CNN to tackle these two problems jointly. We tested our system on
LUNA16 dataset and achieved an average dice similarity coefficient (DSC) of 91%
as segmentation accuracy and a score of nearly 92% for FP reduction. As a proof
of our hypothesis, we showed improvements of segmentation and FP reduction
tasks over two baselines. Our results support that joint training of these two
tasks through a multi-task learning approach improves system performance on
both. We also showed that a semi-supervised approach can be used to overcome
the limitation of lack of labeled data for the 3D segmentation task.Comment: Accepted for publication at IEEE EMBC (40th International Engineering
in Medicine and Biology Conference
Deformable Capsules for Object Detection
In this study, we introduce a new family of capsule networks, deformable
capsules (DeformCaps), to address a very important problem in computer vision:
object detection. We propose two new algorithms associated with our DeformCaps:
a novel capsule structure (SplitCaps), and a novel dynamic routing algorithm
(SE-Routing), which balance computational efficiency with the need for modeling
a large number of objects and classes, which have never been achieved with
capsule networks before. We demonstrate that the proposed methods allow
capsules to efficiently scale-up to large-scale computer vision tasks for the
first time, and create the first-ever capsule network for object detection in
the literature. Our proposed architecture is a one-stage detection framework
and obtains results on MS COCO which are on-par with state-of-the-art one-stage
CNN-based methods, while producing fewer false positive detections,
generalizing to unusual poses/viewpoints of objects
iBARLE: imBalance-Aware Room Layout Estimation
Room layout estimation predicts layouts from a single panorama. It requires
datasets with large-scale and diverse room shapes to train the models. However,
there are significant imbalances in real-world datasets including the
dimensions of layout complexity, camera locations, and variation in scene
appearance. These issues considerably influence the model training performance.
In this work, we propose the imBalance-Aware Room Layout Estimation (iBARLE)
framework to address these issues. iBARLE consists of (1) Appearance Variation
Generation (AVG) module, which promotes visual appearance domain
generalization, (2) Complex Structure Mix-up (CSMix) module, which enhances
generalizability w.r.t. room structure, and (3) a gradient-based layout
objective function, which allows more effective accounting for occlusions in
complex layouts. All modules are jointly trained and help each other to achieve
the best performance. Experiments and ablation studies based on
ZInD~\cite{cruz2021zillow} dataset illustrate that iBARLE has state-of-the-art
performance compared with other layout estimation baselines
Graph-CoVis: GNN-based Multi-view Panorama Global Pose Estimation
In this paper, we address the problem of wide-baseline camera pose estimation
from a group of 360 panoramas under upright-camera assumption. Recent
work has demonstrated the merit of deep-learning for end-to-end direct relative
pose regression in 360 panorama pairs [11]. To exploit the benefits of
multi-view logic in a learning-based framework, we introduce Graph-CoVis, which
non-trivially extends CoVisPose [11] from relative two-view to global
multi-view spherical camera pose estimation. Graph-CoVis is a novel Graph
Neural Network based architecture that jointly learns the co-visible structure
and global motion in an end-to-end and fully-supervised approach. Using the
ZInD [4] dataset, which features real homes presenting wide-baselines,
occlusion, and limited visual overlap, we show that our model performs
competitively to state-of-the-art approaches
The International Workshop on Osteoarthritis Imaging Knee MRI Segmentation Challenge: A Multi-Institute Evaluation and Analysis Framework on a Standardized Dataset
Purpose: To organize a knee MRI segmentation challenge for characterizing the
semantic and clinical efficacy of automatic segmentation methods relevant for
monitoring osteoarthritis progression.
Methods: A dataset partition consisting of 3D knee MRI from 88 subjects at
two timepoints with ground-truth articular (femoral, tibial, patellar)
cartilage and meniscus segmentations was standardized. Challenge submissions
and a majority-vote ensemble were evaluated using Dice score, average symmetric
surface distance, volumetric overlap error, and coefficient of variation on a
hold-out test set. Similarities in network segmentations were evaluated using
pairwise Dice correlations. Articular cartilage thickness was computed per-scan
and longitudinally. Correlation between thickness error and segmentation
metrics was measured using Pearson's coefficient. Two empirical upper bounds
for ensemble performance were computed using combinations of model outputs that
consolidated true positives and true negatives.
Results: Six teams (T1-T6) submitted entries for the challenge. No
significant differences were observed across all segmentation metrics for all
tissues (p=1.0) among the four top-performing networks (T2, T3, T4, T6). Dice
correlations between network pairs were high (>0.85). Per-scan thickness errors
were negligible among T1-T4 (p=0.99) and longitudinal changes showed minimal
bias (<0.03mm). Low correlations (<0.41) were observed between segmentation
metrics and thickness error. The majority-vote ensemble was comparable to top
performing networks (p=1.0). Empirical upper bound performances were similar
for both combinations (p=1.0).
Conclusion: Diverse networks learned to segment the knee similarly where high
segmentation accuracy did not correlate to cartilage thickness accuracy. Voting
ensembles did not outperform individual networks but may help regularize
individual models.Comment: Submitted to Radiology: Artificial Intelligence; Fixed typo